What Do We Know? 

SEEKING EFFECTIVE MATH AND SCIENCE INSTRUCTION 



1 -! THE URBAN INSTITUTE 




February 2005 



The Urban Institute 

Beatriz Chu Clewell 
Clemencia Cosentino de 
Cohen 

Nicole Deterding 
Sarah Manes 
Lisa Tsui 

Campbell-Kibler 
Associates, Inc. 

Patricia B. Campbell 
Lesley Perlman 
Shay N.S. Rao 
Becky Branting 
Lesli Hoey 
Rosa Carson 



As requested by the GE Foundation, the 
main goal of this review was to identify 
mathematics and science curricula as well 
as professional development models at 
the middle and high school levels that 
had been deemed effective based on their 
success in increasing student achievement. 
Because of its emphasis on student achieve- 
ment, this approach differs from that 
typically used at the local level and that 
adopted by other recent studies conducted 
to find "effective" curricula and profes- 
sional development models. 

Historically, curriculum choice at the 
local level has often been made by a com- 
mittee that decides which curriculum to 
adopt based on considerations only periph- 
erally related to student achievement — such 
as state-imposed standards, recommenda- 
tions of others, cost, and presentations by 
publishers' representatives. Choice of pro- 
fessional development models follows a 
similar pattern. Indeed, there has been very 
little else available to guide school districts 
in their curriculum selection process, since 
for most curricula and textbooks the only 
data at hand are publishers' figures on 
the number of adoptions. That has been 
changing. There is a growing movement 
to assess the effectiveness of math and 
science curricula through various meth- 
odologies, including content analyses, 
comparative studies, case studies, and syn- 
thesis studies. 1 And while there have been 
several studies of the effectiveness of pro- 
fessional development practices, very few 
have measured the effects of these practices 
on student achievement. 

The focus of this review, reflecting the 
GE Foundation's interests, was to identify 
current math and science curricula and 
professional development at the middle 
and high school levels that showed evi- 
dence of positive impact on student 
achievement. Our goal was to come up 
with enough math and science curricula 



with relatively credible evaluations to pro- 
vide some choice to districts and schools 
that wanted guidance in selecting a cur- 
riculum and that wished to use effective- 
ness as a selection criterion. 

Evaluation Study Selection 
Criteria Used 

To be included, studies had to have 

1. Rigorous methodological design 
including comparison groups; 

2. Measures of impact on student 
outcomes that included, but were 
not limited to, test scores; and 

3. Comparative data with experi- 
mental or quasi-experimental 
design preferred. 

As can be seen from the criteria above, 
we chose increases in student achievement 
as the measure of effectiveness. 2 We limited 
our review to evaluations that compared 
student achievement outcomes elicited by 
the curriculum being studied with student 
achievement outcomes of another curricu- 
lum. Where data were available, we also 
took into account the content validity of the 
assessments used as well as the statistical 
significance and the effect size of the differ- 
ences reported. Our review made a special 
effort to identify eligible studies that dis- 
aggregated results by performance of stu- 
dent subgroups. Our review of evaluation 
studies of professional development was 
similarly limited to studies that used 
student achievement as the measure of 
effectiveness. 

Professional Development: 

What We Found 

We identified 18 evaluation studies of pro- 
fessional development models that used 
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student achievement outcomes as 

measures of effectiveness. Features of 

these studies can be found in a full 

report of this review. 3 We found that 

■ Providing professional develop- 
ment for teachers of standards- 
based science curricula is 
effective in increasing student 
achievement. There is evidence 
from the research that standards- 
based science programs that pro- 
vide professional development to 
teachers produce higher student 
achievement outcomes than those 
that do not. 

■ Professional development that is 
tied to curriculum, to knowledge 
of subject matter, and/or to how 
students learn the subject is more 
effective in terms of improving 
student achievement than is 
professional development that 
focuses only on teaching behav- 
iors. A number of studies have 
concluded that the content of pro- 
fessional development — what is 
taught — is more important than its 
format — how it is taught — and that 
content should be linked to subject 
matter knowledge, a specific cur- 
riculum, or the process of student 
learning. For example, some pro- 
fessional development focuses on 
training teachers to model specific 
pedagogical approaches and skills 
using generic lesson formats. The 
goal of these efforts is to get teach- 
ers to change the way they teach, 
expecting that this will increase 
student achievement. More effec- 
tive professional development 
focuses on increasing teacher 
knowledge of the subject matter, 
on a specific curriculum, or on 
the student learning process. The 
goal of these types of efforts is to 
change teacher knowledge, ex- 
pecting that this will, in turn, 
both change teacher behavior and 
increase student achievement. 

■ The amount of professional 
development provided is an 
important factor in influencing 



both change in the teaching 
behavior of teachers and change 
in the classroom environment. 

One study found that behavioral 
change was only evident after 
teachers had received a minimum 
of 80 hours of intensive profes- 
sional development; change in 
the classroom environment was 
achieved after 160 hours. 4 It is 
important to note that at the indi- 
vidual level, teacher preparation in 
the content area was an important 
factor in influencing effectiveness. 

■ Some widely held beliefs about 
what constitutes effective profes- 
sional development are not sup- 
ported by research linked to 
student achievement. 

• Myth: Professional development 
should be distributed over the 
school year. While professional 
development in science seems 
to benefit from distributed time, 
the same effect has not been 
found for math. 

• Myth: More is better. There is no 
evidence to support the belief 
that more contact time over and 
above 80 hours and 160 hours 
results in changes in teaching 
behavior or classroom environ- 
ment, respectively. 

• Myth: Classroom visits are neces- 
sary. Visits to classrooms for 
consultation or coaching as part 
of professional development do 
not necessarily result in greater 
student achievement. 

• Myth: Schoohvide programs are 
more effective. Providing services 
to whole schools rather than 
individuals may not be the most 
important element of profes- 
sional development that leads to 
increased student achievement. 

Mathematics and Science 
Curricula: What We Found 

Mathematics Curricula 

Our review netted 89 middle and 

high school curricula, including eight 



mathematics curricula that were 
developed as part of whole school 
reform efforts. A total of 156 studies 
of student mathematics achievement 
with comparison group data were 
found for 18 of the curricula (20 per- 
cent of the total number of curricula 
originally identified). A list of the 
18 mathematics curricula, together 
with overviews of the curricula and 
their impact studies, appears in 
Appendix A. 

The following findings relate to 
methodology: 

■ Most middle and high school 
mathematics curricula do not 
have studies of student achieve- 
ment with comparison groups 
that can be found through litera- 
ture or web searches. Only three 
of the studies found specified the 
curriculum to which the target cur- 
riculum was being compared. The 
rest compared their curriculum to 
some unnamed curriculum, mak- 
ing comparisons across curricula 
impossible. 

■ If students are going to be judged 
on the results of an external test, 
the mathematics curriculum 
selected should cover the areas 
and skills that are included on 
that test (i.e., the test and curricu- 
lum should be aligned). As would 
be expected, students tend to score 
higher on tests focusing on the 
content and skills covered in their 
curriculum. The content and skills 
covered in traditional math cur- 
ricula tend to reflect the content 
and skills covered in most stan- 
dardized and statewide tests. 
Alternatively, the content and 
skills covered in standards-based 
curricula tend to better reflect that 
which is covered in standards- 
based tests. (For more on this 
issue, see the text box, "Inter- 
preting the Results.") 

The following findings relate to 
outcomes and impacts: 
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■ Studies of six of the curricula 
found that students who use the 
curriculum being tested scored 
higher than comparison students 
on both a majority of standard- 
ized and/or state tests used and 
on a majority of the curriculum- 
based tests used. These curricula 
were Cognitive Tutor, Connected 
Mathematics (CMP), Interactive 
Mathematics Program (IMP), 
Prentice Hall: Tools for Success, 
Saxon Math: An Incremental 
Development, and University of 
Chicago School Mathematics 
Project. Because students using 
these curricula scored higher on 
both types of tests, it is reasonable 
to assume that the effectiveness of 
each curriculum is not an effect of 
the type of test used. 

■ It is difficult to determine any 
differential effect of these math 
curricula on girls and boys. Very 
few of the curricula had studies 
that met our criteria and disaggre- 
gated their findings by sex. Among 
those that did, the results were 
inconsistent. 

■ With the exception of Connected 
Mathematics, too few results per 
curriculum were broken out by 
race/ethnicity to allow us to draw 
general conclusions regarding 
racial/ethnic effects. Connected 
Mathematics, however, appeared 
to reduce racial/ethnic gaps. 

Science Curricula 

We identified 80 science curricula at 
the middle and high school levels. 3 

Similar to the mathematics cur- 
riculum, we found seven science 
curricula that had been developed 
specifically as a part of whole school 
reform. A total of 45 studies of stu- 
dent achievement in science were 
found that met our criteria, covering 
21 (or 26 percent) of the total science 
curricula identified. The 21 curricula 
are listed in Table 2, which together 
with brief descriptions of these cur- 



ricula and the results of impact stud- 
ies, is provided in Appendix B. 

The following findings relate to 
methodology: 

■ As with mathematics curricula, 
most middle and high school cur- 
ricula do not have evaluation 
studies of student achievement 
with comparison groups that can 
be found through published liter- 
ature or web searches. However, 
in contrast to the mathematics cur- 
ricula in Table 1 in Appendix A 
(many of which had been the sub- 
jects of multiple studies), most of 
the science curricula in Table 2 in 
Appendix B had only one eval- 
uation or study, usually unpub- 
lished works available through the 
developer. 

The following findings relate to 
outcomes and impacts: 

■ Science curricula based on the 
inquiry approach are consistently 



more effective than traditional 
science curricula as measured by 
student achievement. The prepon- 
derance of evidence provided by 
meta-analyses and evaluations of 
individual curricula seems to con- 
firm that inquiry-based science 
curricula produce larger effects 
on student achievement than do 
the more "traditional" science 
curricula. 6 

■ It is difficult to determine the 
effect of these science curricula 
on different subgroups of stu- 
dents — such as girls, minority 
group members, and urban stu- 
dents. Very few of the curricula 
had studies that met our criteria 
and disaggregated their findings 
by sex, language minority status, 
or urban location. Surprisingly, 
none of the studies reported data 
disaggregated by race/ ethnicity. 

■ Most science curricula include a 
professional development compo 
nent. Inclusion of a professional 
development component as a part 



Interpreting the Results: Validity of Tests Used 
To Measure Student Achievement 

The evaluation studies for mathematics and science curricula on our lists 
used a variety of assessments — some standardized, some state-mandated, 
some self-developed, and some developed by others specifically for stan- 
dards-based curricula. In the case of mathematics curriculum studies, the 
content and skills covered by most standardized achievement tests tend to 
reflect more closely the content and skills covered by traditional mathe- 
matics curriculum than those covered by standards-based curriculum, 
causing them to have better "content validity" for traditional curriculum. 
Many researchers of standards-based curriculum are aware of this and 
develop their own student achievement tests that more accurately test the 
skills and content of standards-based curriculum. 

If students taking one curriculum score higher than others on both 
types of test, there is no question of interpretation. Beyond that, however, 
judgments of efficacy must take into account the content validity of the 
tests in order to determine which type of curriculum is more effective. 
Although science curriculum studies face a similar dilemma, several stud- 
ies comparing the results of standardized tests with those of other forms of 
assessment have found very small differences. The issue of content valid- 
ity, therefore, seems to be less of a concern in judging the effectiveness of 
science curricula. 
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of the curriculum is far more com- 
mon in science than in mathemat- 
ics because science curricula tend 
to be more discretionary and vari- 
able than mathematics curricula. 
Science curriculum developers, 
therefore, may feel the need to pro- 
vide more guidance to teachers 
about the appropriate instructional 
approaches to be used for specific 
curricula. 



What Can We Conclude 
from Our Review? 

The major conclusions of our review 
that should be most useful to those 
wishing to invest in sustainable 
school reform in science and mathe- 
matics can be summarized as follows: 

■ Effective mathematics curricula in 
middle and high school may be 
either traditional or integrative 
(standards-based). 

■ Effective science curricula in 
middle and high school should 
be inquiry-based rather than 
traditional. 

■ Effective professional develop- 
ment programs are those that 
focus on content rather than for- 
mat and that have the following 
features: 

• Content tied to curriculum, 
knowledge of subject matter, 
and/or how students learn a 
subject; 

• A minimum of 80 contact hours 
to effect changes in teachers' 
instructional behaviors; and 

• A minimum of 160 contact 
hours to effect changes in the 
classroom environment. 

How to Use This Review 

Choice of a curriculum in mathe- 
matics and science involves deciding 
what aspects of these subjects are 
important to address and emphasize 
in schools — this choice thus deter- 



mines what students will learn. Once 
a decision is made, selection should 
be guided by how effective a par- 
ticular curriculum is for the student 
population to be taught. The National 
Research Council report. On Eval- 
uating Curricular Effectiveness: Judging 
the Quality ofK-12 Mathematics 
Evaluations (2004) describes the value 
of this knowledge to the decision- 
making process: 

Clearly, knowing how effective a 
particular curriculum is, and for 
whom and under what conditions it 
is effective, represents a valuable 
and irreplaceable source of informa- 
tion to decisionmakers, whether 
they are classroom teachers, par- 
ents, district curriculum specialists, 
school boards, state adoption 
boards, curriculum writers and 
evaluators, or national policymak- 
ers. Evaluation studies can provide 
that information but only if those 
evaluations meet standards of qual- 
ity (p.l)7 

This review of math and science 
curricula has tried to simplify for 
schools and districts the complex, 
time-consuming process of determin- 
ing curriculum effectiveness by iden- 
tifying programs that have what we 
consider to be credible evaluations. 
We have also distilled the findings of 
achievement-based research on pro- 
fessional development to a handful of 
principles that reflect effective prac- 
tice. Once schools and districts have 
decided on a curriculum and an 
appropriate assessment tool, they 
might wish to collect their own 
impact data to evaluate how well the 
curriculum they choose is working 
with their own students. 8 It is the 
responsibility of the school and dis- 
trict community to ensure that the 
content that they want students to 
learn is embodied in the curriculum, 
that the curriculum is effective for 
this purpose, and that appropriate 
measures are used to assess whether 
students are indeed learning what the 
community wants them to learn. It is 



an enormous task and an enormous 
responsibility. We hope that our 
review can provide some guidance 
and assistance in the process. 

Endnotes 

1. The majority of these efforts have been 
undertaken by A A AS, the National Research 
Council (NRC), and the U.S. Department of 
Education. The AAAS study used content 
analysis, the NRC study did not rate specific 
curricular math programs, and the U.S. 
Department of Education study reviewed 
middle school math programs only. 

2. As determined by quantifiably measurable 
outcomes, such as test scores. 

3. The full report, on which this summary 
report is based and which contains refer- 
ences and a bibliography of all studies 
described in this report, can be found on 
the Urban Institute web site, http://www. 
urban.org. 

4. "Teacher behavior" in this context refers 
to teachers' use of specific pedagogical 
approaches in instruction, such as inquiry- 
based teaching practices. "Change in the 
classroom environment" refers to teacher 
facilitation of an investigative classroom cul- 
ture through seating arrangements to stimu- 
late discussion, use of cooperative learning 
groups, encouraging students to explain 
concepts to one another, and other such 
practices. 

5. Of the 80 science curricula identified, 59 had 
no evaluations or had evaluations that did 
not meet our criteria, or evaluations were 
out of print, or we did not receive a re- 
sponse from the developers. 

6. As measured by both standardized tests and 
tests developed for standards-based curric- 
ula in both meta-analyses and evaluations of 
individual curricula. 

7. Jere Confrey and Vicki Stohl, eds. 2004. On 
Evaluating Curricular Effectiveness: Judging the 
Quality ofK-12 Mathematics Evaluations. 
Committee for a Review of the Evaluation 
Data on the Effectiveness of NSF-Supported 
and Commercially Generated Mathematics 
Curriculum Materials. National Research 
Council. Washington, DC: National Acad- 
emy Press. 

8. For those schools and districts that already 
have programs that they feel are effective 
but are not included in Tables 1 and 2, we 
suggest that they collect their own effective- 
ness data via evaluation studies that adhere 
to the criteria established in this review. 

They may wish to contract with an evalua- 
tor for this purpose. 
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Appendix A 

DETAILED FINDINGS OF THE REVIEW OF MATH CURRICULA 

This appendix provides a list of the mathematics curricula included in our review, together with a descrip- 
tion of each curriculum and its related evaluation studies. Table 1 lists the mathematics curricula identified 
as having evaluation studies that met our criteria. 



TABLE 1. Math Curricula with Studies 




Note: Shaded curricula are those for which we have found the strongest evidence of effectiveness, that is, quantitative evidence that their use in instruction elicits higher achieve- 
ment/performance in students than other curricula to which they are compared on both standardized and/or state tests and on curriculum developed tests. There are several curricula 
for which this evidence of effectiveness has not been collected but which might also qualify as effective should appropriate studies be conducted. An omission from this list of many 
curricula signifies merely that these curricula have not yet provided quantitative evidence of effectiveness that meets our criteria. 

*An asterisk marks curricula for which effectiveness data are provided for subgroups of students, indicated in brackets. (See full report on the web for discussion of research find- 
ings by subgroup.) 
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A Description of the Mathematics Curricula 
and Evaluation Studies Found 

In addition to a description of each curriculum, these 
overviews (presented in alphabetical order) include the 
type of student achievement measure used, the number 
and direction of the results, and, if available, the size of 
any differences between groups. If the results were broken 
out by sex and/or race and ethnicity, this too is indicated. 

The types of measures used in the impact studies are 
broken into five categories: 

■ Standardized achievement tests (i.e., PSAT, SAT, SAT-9, 
and Iowa Test of Basic Skills) 

■ State-mandated achievement tests (i.e., FCAS, MEAP, 
and MCAS) 

■ Standards-based, curriculum-driven measures (i.e.. 
Balanced Assessment, Problem Situation Test) 

■ Teacher-based measures (i.e., GPA, school tests, and 
mathematics courses taken) 

■ Percent passing different mathematics courses 

In the charts that follow within this appendix, results 
were tracked rather than the number of studies. Multiple 
grades and multiple measures were counted as multiple 
results and broken out as such. For example, a study that 
looked at the impact of different subsets of a curriculum 
for sixth, seventh, and eighth grades was counted as three 
results. Cohort studies that tracked students across multi- 
ple grades were counted as one result. Also counted as one 
result were findings from different sites within the same 
study using the same measure. If a majority of sites had 
changes favoring the tested curriculum, the result was 
indicated as positive. If the majority of sites did not differ 
from the comparison, the result was indicated as no 
change. If a majority of sites had changes favoring the 
comparison curriculum, the result was indicated as 
negative. 

Note: Most impact studies for math curricula reported the statistical sig- 
nificance of their results. Only differences that have reached the conserva- 
tive minimum acceptable statistical significance level of .05 were included 
in the results reported for each study. If differences are statistically signifi- 
cant, then there is another measure, called an effect size, which shows 
how big the difference is. In our description of the study results, we pro- 
vide effect sizes where available, although very few studies reported 
these results. Effect sizes greater than .4 are considered large; between .2 
and .4 are considered moderate; and less than .2 are considered small. 



Advanced Placement (AP) Calculus 

The AP Calculus curriculum includes two courses: AP 
Calculus AB, which is comparable to one semester of col- 
lege-level calculus, and AP Calculus BC, which is com- 
parable to two semesters of college-level calculus. Both 
courses include elementary functions, limits and continu- 



ity, and differential and integral calculus, with Calculus BC 
covering these topics more extensively than Calculus AB. 
Prerequisites include knowledge of analytic geometry and 
elementary functions in addition to college preparatory 
algebra, geometry, and trigonometry. Enrolled students are 
expected to take the Advanced Placement examination in 
Calculus AB or BC. 

Contact 

College Board Headquarters 
45 Columbus Avenue 
New York, NY 10023-6992 
Tel: 212-713-8066 

Web site: http://apcentral.collegeboard.eom/article/0, 3045, 
151-165-0-2178,00.html 

Results 

Five results were found from studies on the effect of AP 
Calculus courses. These studies compared college mathe- 
matics performance of students who took AP Calculus 
with other students who did not. 



Type of measure 


Number 
of results 


Results 


Effect size 


Teacher-based 

measures/GPA 


3 


AP students scored higher 
in two results; there were 
no differences in one 
result. 


No effect 
size reported 


Pass rates/ 
courses taken 


2 


AP students scored higher 
in one result, and there 
were no differences in 
one result. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



Cognitive Tutor 

The Cognitive Tutor (CT), from Carnegie Learning, 
includes full curricula in Algebra I, Geometry, Algebra II, 
an Integrated Math Series, and Quantitative Literacy 
Through Algebra. Each curriculum combines software- 
based, individualized computer lessons with collaborative, 
real-world problem-solving activities. Students spend 
about 40 percent of their class time using the software and 
the remainder of their time engaged in classroom problem- 
solving activities. 

Contact 

Carnegie Learning, Inc. 

1200 Penn Avenue 
Suite 150 

Pittsburgh, PA 15222 
Tel: 888-851-7094 

E-mail: info@carnegielearning.com 
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Results 

Twenty-one results from studies on the effect of CT were 
found, all of which focused on Algebra I at the middle and 
high school level. 



Results 

Six results from studies on the effect of CPM were found, 
one of which focused on algebra and five of which focused 
on grade 9-11 math. 



Number 

Type of measure of results 


Results 


Effect size 


Standardized 

achievement 

tests 


8 


CT students scored higher 
in all eight results. 


Moderate 
effect size 
in one result 


Statewide tests 


5 


CT students scored higher 
in three results; there 
were no differences in 
two results. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


7 


CT students scored higher 
in seven results. 


Large effect 
sizes in two 
results 


Passing rates 


1 


CT students had higher 
passing rates. 


No effect 
size reported 



Two results looked at sex differences; in one result, CT 
boys' scores were higher than comparison boys. There 
were no differences between boys in one result, and there 
were no differences for the girls' scores in either result. 
Two results reported differences by race and ethnicity. In 
one result, African-American CT students had higher 
scores than comparison African-American students, and 
there were no differences for Hispanic students. In the 
second result, Hispanic CT students scored higher than 
comparison Hispanic students. 



College Preparatory Mathematics (CPM) 

The CPM series offers a four-year integrated curriculum 
where mathematics topics are revisited and built upon 
through the years. Problem-solving strategies are empha- 
sized as a vehicle for learning mathematics, and student 
study teams are an integral part of the learning process. 
Based on the belief that concept mastery requires time, the 
curriculum spirals through practice of the main course 
concepts throughout each year and emphasizes students' 
supportive group work. The program sees the teacher's 
role as a guide. 

Contact 

CPM Business Office 
1233 Noonan Drive 
Sacramento, CA 95822 
Tel: 916-681-3611 
Web site: http://www.cpm.org 





Number 






Type of measure 


of results 


Results 


Effect size 


Standardized 


5 


There were no differences 


No effect 


achievement 

tests 




in all five results. 


size reported 


Teacher- 


1 


There were no differences. 


No effect 


developed 






size reported 


measure 









Results were not reported by sex, race, or ethnicity. 



Connected Mathematics (CMP) 

The Connected Mathematics (CMP) curriculum is com- 
posed of eight models, each focusing on one important 
area of mathematics and emphasizing previously learned 
content. Connected Mathematics is designed to develop 
students' knowledge and understanding of mathematics 
through attention to connections between mathematical 
ideas and their applications in the world outside school; 
among the core ideas in mathematics; among the strands 
in a modem mathematics curriculum; and between the 
planned teaching-learning activities and the special apti- 
tudes and interests of middle school students. 

Contact 

Pearson Education 
P.O. Box 2500 
Lebanon, IN 46052-3009 
Tel: 800-848-9500 

Web site: http://www.phschool.com/math/ 



Results 

Thirty-four results from comparison studies looking at 
the effect of CMP were found focusing on middle grade 
mathematics. 



Number 

Type of measure of results Results Effect size 

Standardized 11 CMP students scored No effect 
achievement higher for six results; they size reported 

tests scored lower for one result; 

and there were no differ- 
ences for four results. 

(i continued ) 
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Number 

Type of measure of results 


Results 


Effect size 


Statewide 

tests 


16 


CMP students scored 
higher for 14 results; there 
were no differences for 
two results. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


6 


CMP students scored 
higher for all six results. 


Large effect 
sizes were 
found for 
one result. 


Teacher-based 

measure 


1 


CMP students scored 
higher in one result. 


No effect 
size reported 



Seven results were broken out by race/ ethnicity and 
two by sex. No significant sex differences were found. 
African-American CMP students were found to score 
higher than African-American comparison students in six 
results, while Hispanic CMP students scored higher than 
Hispanic comparison students in four results; in a fifth 
result, there were no differences. African-American and 
Hispanic CMP students showed greater gains than others 
in two results, while African Americans alone showed 
greater gains in one result. In one result. Native- American 
CMP student performance decreased. 



Contemporary Mathematics in Context: 

A Unified Approach 

(Core PLUS Mathematics CPMP) 

Core-Plus Mathematics consists of a single core sequence 
for both college-bound and employment-bound students 
during the first three years of high school. A flexible fourth 
year course can be used to prepare students for college 
mathematics. 

Contact 

Core-Plus Mathematics Project 
Department of Mathematics 
Western Michigan University 
Kalamazoo, Ml 49008-5248 
Tel: 866-407-CPMP (2767) 

E-mail: cpmp@wmich.edu 

Results 

Fifty-six results from comparison studies on the effect of 
CPMP were found focusing on high school math. 



Number 



Type of measure of results 


Results 


Effect size 


Standardized 

achievement 

tests 


43 


In 1 9 of the results, CPMP 
students scored higher; 
in 24 results, there were 
no differences. 


No effect 
size reported 


Statewide 

tests 


5 


Students scored higher in 
all five results. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


2 


Students scored higher in 
both results. 


No effect 
size reported 


Teacher-based 

measure/GPA 


6 


CPMP students scored 
higher in one result, no 
differently in another, and 
lower in four results. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



Direct Instruction 

In Direct Instruction (DI), each program is fully scripted, 
from what the teacher says and anticipated student 
responses, to correctional procedures. Each skill is broken 
down into its component parts, and then each component 
of the skill is taught to mastery. Afterward, the skills are 
combined within a larger context where they may be uti- 
lized across settings, resulting in generalized fluency. The 
DI mathematics curriculum covers grades K-8, but focuses 
primarily on grades K-6. The mathematics curriculum 
covers 19 different topics ranging from addition and sub- 
traction to money, mathematics study skills (graphs, 
charts, maps, and statistics), and geometry. 

Contact 

The McGraw-Hill Companies 
P.O. Box 182604 
Columbus, OH 43272 
Tel: 888-772-4543 

Web site: http://www.sraonline.com/index.php/home/ 
curriculumsolutions/di/connectingmath/1 14 

Results 

Nine results from comparison studies on the effect of DI 
were found which focus on seventh and eighth grade 
math. 
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Number 



Type of measure 


of results 


Results 


Effect size 


Standardized 


6 


Dl students scored higher 


No effect 


achievement 

tests 




for four results; they 
scored lower for two 
results. 


size reported 


State test 


3 


In all three results, Dl 
students scored higher. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



Edison Schools 

Founded in 1992, Edison Schools is focused on raising stu- 
dent achievement through research-based school design, 
aligned assessment systems, interactive professional devel- 
opment, integrated use of technology, and other program 
features, including a longer school day and school year. The 
math curriculum for grades 6-8 includes applied arithmetic, 
prealgebra, and pregeometry, using a spiral curriculum 
approach to teach concepts and ideas. The mathematics in 
grades 9-10 provides three years of high school math in two 
years' time, using an integrated application-based approach 
to algebra, geometry, and trigonometry with additional 
emphasis on probability, statistics, and discrete mathemat- 
ics. Grades 11 and 12 incorporate advanced mathematical 
modeling. Calculus and Statistics, Algebra 2, and 
Precalculus, as well as AP Statistics and Calculus. 

Contact 

521 Fifth Avenue, 11th Floor 
New York, NY 10175 
Tel: 212-419-1600 

Web site: http://www.edisonschools.com/ 

Results 

Seventeen results from comparison studies on the effect of 
Edison Schools were found; fifteen focus on middle school 



math and two focus on high school math. 


Type of measure 


Number 
of results 


Results 


Effect size 


Standardized 

achievement 

tests 


8 


For four results, Edison 
students scored higher; 
in the remaining four, there 
were no differences. 


No effect 
size reported 


State test 


9 


For two results, Edison 
students scored higher; 
for four results, Edison 
students scored lower; 
and for three results, there 
were no differences. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



Integrated Mathematics, Science, 
and Technology (IMaST) 

The Integrated Mathematics, Science, and Technology pro- 
gram provides integrated sixth, seventh, and eighth grade 
curricula that promote hands-on learning for students and 
teamwork among teachers from different disciplines. 
IMaST emphasizes learning based on constructivist theory 
and active student participation involving a hands-on 
approach comprising a wide variety of activities. 

Contact 

Ronjon Publishing, Inc. 

1001 S. Mayhill Rd. 

Denton, TX 76208 
Tel: 800-262-3060 
Web site: 

http://www.ilstu.edu/depts/cemast/programs/imast.shtml 

Results 

One study was found looking at the effect of IMaST focus- 
ing on seventh and eighth grade students. 





Number 






Type of measure 


of results 


Results 


Effect size 


Standardized 


1 


IMaST students scored 


No effect 


achievement 

tests 




higher for the one result. 


size reported 



Results were not reported by sex, race, or ethnicity. 



Interactive Mathematics Program (IMP) 

This four year, problem-based curriculum incorporates tra- 
ditional branches of mathematics (algebra, geometry, and 
trigonometry) with additional topics recommended by the 
NCTM Standards, such as statistics, probability, curve fit- 
ting, and matrix algebra. Students are encouraged to 
experiment, investigate, ask questions, make and test con- 
jectures, reflect, and accurately communicate their ideas 
and conclusions. Although each unit has a specific mathe- 
matical focus, other topics are brought in as needed to 
solve the central problem. Ideas that are developed in one 
unit are usually revisited and deepened in one or more 
later units. Algebra and geometry are distributed through- 
out the four years. 
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Contact 

Key Curriculum Press 
1150 65th Street 
Emeryville, CA 94608 
Tel: 800-995-MATH (6824) 

Web site: http://www.keypress.com/catalog/products/ 
textbooks/Prod JMP.html 



Results 

Twenty-two results from comparison studies were found 
on the effect of IMP, all focusing on high school. 



Number 

Type of measure of results 


Results Effect size 


Standardized 

achievement 

tests 


12 


IMP students scored No effect 

higher for eight results; size reported 

there were no differences 
for four results. 


Pass rates or 
courses taken 


5 


In all five results, IMP No effect 

students had higher pass- size reported 
ing rates and/or were more 
likely to take more mathe- 
matics courses. 


Curriculum-driven, 
skill-specific tests 


3 


IMP students scored Large effect 

higher for all three results, sizes were 
found in the 
three results 


Teacher-based 

measure 


2 


In both results, IMP No effect 

students scored higher. size reported 



Results were not reported by race or ethnicity. The one 
result reported by sex found IMP girls were slightly more 
apt to continue three or more years in math than IMP boys, 
while the reverse was the case for comparison students. 



MATH Connections 

MATH Connections (MC) is an integrated curriculum that 
blends ideas from traditionally separate mathematical 
fields (e.g., algebra, geometry, statistics, and discrete math- 
ematics) in ways that blur the lines between them. This 
three year curriculum replaces the traditional Algebra I, 
Geometry, Algebra II sequence and is designed for all stu- 
dents in grades 9, 10, and 11, with honors students begin- 
ning the curriculum in grade 8. 

Contact 

750 Old Main Street 
Suite 303 

Rocky Hill, CT 06067-1567 
Tel: 860-721-7010 

Web site: http://www.mathconnections.com 



Results 

Fifteen results from comparison studies were found on the 
effect of MC, all of which focused on high school students. 



Type of measure 


Number 
of results 


Results 


Effect size 


Standardized 

achievement 

tests 


6 


MC students scored 
higher for three results; for 
three results, there were 
were no differences. 


No effect 
size reported 


State test 


8 


MC students scored 
higher for all eight results. 


There was a 
large effect 
size for three 
results. 


Curriculum-driven, 1 
skill-specific test 


No difference was found. 


N/A 



One result looked at sex, race, and ethnic differences 
and found no differences within MC students. 



Mathematics in Context (MiC) 

Mathematics in Context is a four-year middle school cur- 
riculum (grades 5-8) that encourages students to discover 
mathematical concepts and skills through engaging prob- 
lems and meaningful contexts. Each year includes lessons 
in the four strands (numbers, algebra, geometry and statis- 
tics, and probability) that are interwoven through 10 units. 
For example, sample algebra units include Patterns and 
Symbols (grade 5), Expressions and Formulas (grade 6), 
Ups and Downs (grade 7), and Graphing Equations 
(grade 8). 

Contact 

Holt, Rinehart and Winston 

Attn: Ms. Webl 

10801 N. MoPac Expressway 

Building 3 

Austin, TX 78759 

Tel: 800-HRW-9799 (800-479-9799) 

Web sites: http://www.hrw.com 

http://mic.britannica.com/mic/common/home.asp 

Results 

Twenty results from comparison studies were found on the 
effect of MiC, which focused on middle school students. 
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Type of measure 


Number 
of results 


Results 


Effect size 


Standardized 


17 


In 15 results, MiC 


In two 


achievement 




students scored higher; in 


results, a 


tests 




two results, there were no 
differences. 


majority of 
classes 
showed at 
least moder- 
ate effect 
sizes. 


State test 


3 


MiC students scored 
higher for all three results. 


No effect 
size reported 



One result was broken out by race. No statistically sig- 
nificant difference was found between African-American 
students using MiC and comparison students. 



Mathematics: Modeling Our World 
(MMOW/ARISE) 

In Mathematics: Modeling Our World (MMOW), students 
are taught to use a variety of resources to solve problems 
and to choose resources that meet the needs of a particular 
situation. As in real life, MMOW's problems do not neces- 
sarily have perfect solutions. MMOW works to strengthen 
the students' ability to solve problems by setting goals and 
thinking strategically about how to achieve these goals, 
solving problems through trial and error and / or process 
of elimination, using technology like calculators and com- 
puters, and working together to solve semi-structured 
problems and communicating the solutions. 

Contact 

W.H. Freeman and Company 
41 Madison Avenue 
New York, NY 10010 
Tel: 800-446-8923 

Web sites: http://www.whfreeman.com/highschool/ 
contact_hs_rep.asp 

http://www.comap.com/highschool/projects/mmow/ 

introduction.htm 

Results 

Four results related to MMOW were found, one focusing 
on middle school students and three focusing on high 
school students. 



Number 

Type of measure of results Results Effect size 

Standardized 4 For three results, MMOW No effect 
achievement students scored higher; size reported 

tests for one result, there were 

no differences. 

Results were not reported by sex, race, or ethnicity. 



Mathematics with Meaning 

Mathematics with Meaning (MwM) is not a complete cur- 
ricular program; rather, it is a combination of professional 
development, instructional strategies, and carefully 
planned materials designed to alter the pedagogy and 
content of middle school and high school mathematics 
courses in order to improve student achievement. The pro- 
gram consists of instructional units that teachers may use 
on a supplementary basis or as their entire instructional 
program. MwM takes a student-centered approach based 
on exploratory learning and problem solving, focusing on 
developing conceptual understanding, connections, and 
communication with mathematical concepts through fre- 
quent group work and hands-on activities. 

Contact 
College Board 
Dept CBO 
P.O. Box 869010 
Plano, TX 75074 
Tel: 212-713-8260 
Tel: 800-323-7155 

E-mail: Collegeboardcustomerservice@pfsweb.com 
Web site: http://www.collegeboard.com 

Results 

Eleven results from comparison studies on the effect of 
MwM were found; three focused on middle school math 
achievement and eight on high school math achievement. 





Number 






Type of measure 


of results 


Results 


Effect size 


Statewide tests 


11 


In six results, there were 


No effect 






no differences; in five 
results, MwM students 
scored higher. 


reported 



Six results were broken out by sex, four by race. In the 
six results where sex differences were given, boys slightly 
outperformed girls. In four results where race /ethnic dif- 
ferences were given, achievement scores were significantly 
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lower for African-American students than other students 
in both MwM and comparison groups. 



Middle Grades MATHThematics 

Middle Grades MATHThematics (STEM) is a three-year 
curriculum designed for use in grades 6-8. Four unifying 
concepts — Proportional Reasoning, Multiple Represen- 
tations, Patterns and Generalizations, and Modeling — are 
used across the three years with seven content strands: 
Number, Measurement, Geometry, Statistics, Probability, 
Algebra, and Discrete Mathematics. 

Contact 

McDougal Littell Customer Service Center 

A Houghton Mifflin Company 

1900 S. Batavia 

Geneva IL 60134 

Tel: 617-351-5326 

Tel: 800-462-6595 

Web sites: http://www.mcdougallittell.com/ 
http://www.classzone.com/math_middle.cfm 



Results 

Six results from comparison studies were found on the 
effect of STEM. 



Number 

Type of measure of results 


Results 


Effect size 


Standardized 

achievement 

tests 


2 


In one result, STEM stu- 
dents scored higher; in 
the second, there were no 
differences. 


No effect 
size reported 


Statewide tests 


2 


In one result, STEM stu- 
dents scored higher; in 
the second, there were no 
differences. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


2 


In both results, STEM stu- 
dents scored higher. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



Prentice Hall: Tools for Success 

In Prentice Hall Math (PHM), various mathematical 
strands (such as number sense, algebra, geometry, mea- 
surement, data analysis, and problem solving) are inte- 
grated throughout the series to ensure that students are 
prepared for subsequent mathematics courses at the high 



school level. Each lesson includes a Think and Discuss sec- 
tion that presents the new material along with questions to 
get students actively thinking about and discussing impor- 
tant concepts. Textbooks in this curriculum include Work 
Together activities that allow students to work in groups, 
often doing hands-on activities to reinforce math topics. 

Contact 

Pearson Education 
P.0. Box 2500 
Lebanon, IN 46052-3009 
Tel: 800-848-9500 

Web site: http://www.phschool.com/math/ 



Results 

Eleven results from comparison studies on the effect of 
PHM were found. 



Number 

Type of measure of results 


Results 


Effect size 


Standardized 

achievement 

tests 


8 


PHM students scored 
higher in five results; in 
three results, there 
were no differences. 


Moderate 
effect sizes 
in one result 


Statewide tests 


2 


PHM students scored 
higher in both results. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


1 


PHM students scored 
higher in this result. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



Saxon Math: An Incremental Development 

Saxon Math (SM) is a K-12 curriculum that systematically 
distributes instruction, practice, and assessment through- 
out the academic year rather than concentrating concepts 
in a single unit or chapter. Each increment builds upon the 
foundation of earlier increments, to lead students toward 
a deeper understanding of mathematical concepts. 
Instruction of related concepts is spread throughout the 
grade level, ensuring that students have an opportunity 
to master each concept before they are introduced to the 
next one. 

Contact 

Saxon Publishers 
2600 John Saxon Blvd. 

Norman, OK 73071 
Tel: 800-284-7019 

E-mail: info@saxonpublishers.com 
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Results 

A total of eight results from comparison studies were 
found on the impact of SM, focusing on both middle and 
high school. 



Number 

Type of measure of results 


Results 


Effect size 


Standardized 


3 


SM students scored 


No effect 


achievement 

tests 




higher in all three results. 


size reported 


Statewide tests 


3 


SM students scored 
higher in all three results. 


No effect 
size reported 


Curriculum-driven, 


1 


SM students scored 


No effect 


skill-specific tests 




higher in this result. 


size reported 


Teacher-based 


1 


SM students scored 


No effect 


measure/GPA 




higher in this result. 


size reported 



Results were not reported by sex, race, or ethnicity. 



Systemic Initiative for Montana 
Mathematics and Science (SIMMS) 

The SIMMS curriculum is divided into six levels, each 
consisting of one year of work. Level 1 is typically offered 
to ninth graders, followed by level 2 in grade 10. After 
completing level 2, students may choose between levels 3 
and 4 and then proceed to either level 5 or 6 in the subse- 
quent year. The sequence for potential math and science 
majors is 1-2-4-6. Levels 1 and 2 offer basic mathematical 
literacy. 

Contact 

Kendall/Hunt Publishing 
4050 Westmark Drive 
P.O. Box 1840 
Dubuque, IA 52004-1840 
Tel: 800-542-6657 

Web sites: http://www.simms-im.com 
http://www.montana.edu/~wwwsimms/ 

Results 

Twelve results from comparison studies were found on the 
effect of SIMMS. Four results each focused on levels 1 and 
2, and two each focused on levels 4 and 6. 



Number 



Type of measure of results 


Results 


Effect size 


Standardized 

achievement 

tests 


6 


No differences were found 
in the six results. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


6 


In one result, SIMMS stu- 
dents scored higher; no 
differences were found in 
the other five. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 



University of Chicago School Mathematics 
Project (UCSMP) 

The UCSMP secondary curriculum consists of six courses: 
Transitional Mathematics; Algebra; Geometry; Advanced 
Algebra; Functions, Statistics, and Trigonometry; and Pre- 
Calculus and Discrete Mathematics. Transitional Mathe- 
matics, which was originally designed for average to above 
average seventh graders (but can be started earlier or later), 
weaves together three more or less equal strands of major 
content: applied arithmetic, prealgebra, and elementary 
geometry. Algebra, geometry, and some discrete mathematics 
are integrated into all courses, as are statistics and probability. 

Contact 

UCSMP 

5835 South Kimbark Avenue 
Chicago, IL 60637 
Tel: 773-702-1130 

Web site: http://socialsciences.uchicago.edu/ucsmp/ 

Results 

Fourteen results were found from comparison studies 
on the effect of UCSMP. The following UCSMP courses 
were covered: Transitional Mathematics (2), Algebra (2), 
Geometry (5), Advanced Algebra (4), and Pre-Calculus and 
Discrete Mathematics (1). 



Number 



Type of measure of results 


Results 


Effect size 


Standardized 

achievement 

tests 


5 


In all five results, UCSMP 
students scored higher. 


No effect 
size reported 


Curriculum-driven, 
skill-specific tests 


8 


In all eight results, UCSMP 
students scored higher. 


No effect 
size reported 


Teacher-based 

measure/GPA 


1 


No differences were found. 


No effect 
size reported 



Results were not reported by sex, race, or ethnicity. 
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Appendix B 

DETAILED FINDINGS OF THE REVIEW OF SCIENCE CURRICULA 

This appendix provides a list of the science curricula included in our review, together with a description of 
each curriculum and its related evaluation studies. Table 2 below lists the science curricula that we identified 
as having evaluation studies that met our criteria. 



TABLE 2. Science Curricula with Studies 



Grades Covered 



K 12 



K 8 



PK 8 



K 6 



2 8 



6 8 



6 8 



6 8 



6 8 



6 8 



6 8 



6 9 



7 8 



7 9 



7 9 



7 9 



9 11 



9 12 



9 12 



9 12 



10 12 



Curriculum Name 



Expeditionary Learning Outward Bound (ELOB) *[LEP] 

Full Option Science System (FOSS) *[LEP] 

Great Explorations in Math and Science (GEMS) 

The Science Curriculum Improvement Study (SCIS) 

National Science Curriculum for High Ability Learners 
DESIGNS/DESIGNS II *[sex] 

Integrated Math, Science and Technology (IMaST) 

Center for Learning Technologies in Urban Schools (LeTUS) *[sex] 
Learning By Design (LBD) 

Science 2000/Science 2000+ 



Science and Technology Concepts for Middle School (STC/MS) 

Event-Based Science (EBS) 

Constructing Ideas in Physical Science (CIPS) *[sex] 

Foundational Approaches in Science Teaching (FAST) 

Global Lab Curriculum (GLC) 

Issues, Evidence and You (IEY)/SEPUP 
BSCS: An Inquiry Approach 
High Schools that Work (HSTW) 

Modeling Instruction in High School Physics *[sex] 

World Watcher/Learning About the Environment (LATE) * [urban] 

Physics Resources and Instructional Strategies for Motivating Students (PRISMS) 



Subject Matter 

Whole School Reform 

Multi-Science 

Math & Multi-Science 

Multi-Science 

Multi-Science 

Physical Science 

Math, Science & Technology 

Multi-Science 

Multi-Science 

Multi-Science 

Multi-Science 

Earth Science 

Physical Science 

Multi-Science 

Multi-Science, Environmental Focus 

Multi-Science 

Multi-Science 

Whole School Reform 

Physics 

Environmental Science 
Physics 



Note: Shaded curricula are those for which we have found the strongest evidence of effectiveness, that is, quantitative evidence that (1 ) their use in instruction elicits higher achieve- 
ment/performance in students than other curricula to which they are compared on both standardized and/or state tests and on curriculum developed tests, or (2) they showed large 
effect sizes in terms of increasing student achievement. All science curricula listed in Table 2, however, have credible evaluations that show evidence of effectiveness. There are sev- 
eral curricula for which this evidence of effectiveness has not been collected but which might also qualify as effective should appropriate studies be conducted. An omission from 
this list of many curricula signifies merely that these curricula have not yet provided quantitative evidence of effectiveness that meets our criteria. 

*An asterisk marks curricula for which effectiveness data are provided for subgroups of students, indicated in brackets. (See full report on the web for discussion of research find- 
ings by subgroup.) 
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A Description of the Science Curricula 
and Evaluation Studies Found 

The attached descriptions of science curricula, which 
appear in alphabetical order, include the type of student 
achievement measure used, the number and direction of 
the results, and, if available, the size of any differences 
between groups. When known, effect sizes have been 
listed. Where results have been provided by race/ ethnic- 
ity, sex, or other demographic characteristics, we have re- 
ported these. The absence of such notation means that no 
data were reported by subgroup. In the case of three cur- 
ricula, descriptions of studies do not follow the normal 
format because results reported are not best described in 
that particular format. 

Note: Almost all impact studies for science curricula reported the statisti- 
cal significance of their results. Only differences that have reached the 
conservative minimum acceptable statistical significance level of .05 were 
included in the results reported for each study. If differences are statisti- 
cally significant, then there is another measure, called an effect size, that 
shows how big the difference is. In our description of the study results, 
we provide effect sizes where available, although few studies reported 
these results. Effect sizes greater than .4 are considered large, between .2 
and .4 are considered moderate, and less than .2 are considered small. 



BSCS: An Inquiry Approach 

This program introduces 9th, 10th, and 11th grade stu- 
dents to the core concepts in inquiry, the physical sciences, 
the life sciences, and the earth-space sciences as articu- 
lated in the National Science Education Standards. In addi- 
tion, the curriculum engages students in integration across 
the disciplines in relevant contexts that explore the stan- 
dards related to science in a personal and social perspec- 
tive. This program provides high school students with an 
alternative to the traditional sequence of biology, chem- 
istry, and physics. Included with this program is a profes- 
sional development component designed to help teachers 
and school districts implement the materials. 



Type of measure 


Number 
of results 


Results 


Effect size 


Content test, 


1 


Average student gains at 


No effect 


unknown if self- 




both 9th and 1 0th grade 


size reported 


designed or 




levels were between 20 




state-issued 




and 25 percent. 





Note: Statistical significance levels were not reported. 



The field test was conducted across 10 states and 
included students in urban, suburban, and rural schools. 
However, data regarding specific demographic groups 
was not provided. 



Center for Learning Technologies 
in Urban Schools (LeTUS) 

LeTUS, developed by researchers at the University of 
Michigan and Detroit public school teachers, includes pro- 
ject-based curriculum materials that build from district, 
state, and national standards to support the development 
of integrated science understanding for middle school stu- 
dents. The materials support students' science learning 
through engaging them in inquiry about real world prob- 
lems, providing them with multiple opportunities to work 
with concepts, and integrating the use of learning tech- 
nologies in instruction. LeTUS is focused on learning 
about and developing a new machine to construct large 
buildings and bridges, an area that has been identified as 
of interest to young urban students. 

Contact 

Joseph Krajcik 

School of Education 

University of Michigan 

610 East University Avenue, Rm. 4109 

Ann Arbor, Ml 48109-1259 

Tel: 734-647-0597 

Fax: 734-615-5245 

E-mail: krajcik@umich.edu 



Contact 

Pamela Van Scotter 
Director 

The BSCS Center for Curriculum Development 
BSCS 5415, Mark Dabling Blvd. 

Colorado Springs, Colorado 80918 
Web site: http://www.bscs. org/page.asp?id=curriculum_ 
development! high_school_9-1 21 An_lnquiry_Approach 

Results 

Most of the evaluative work on this was of the materials, 
not student achievement; however, the results of a nation- 
wide field test are summarized below. 



Results 

Two evaluations of the LeTUS program were found, one 
that utilized skill-specific instruments and one that used 
statewide achievement test scores to measure effectiveness. 



Type of Number 

measure of results Results Effect size 



Curriculum- 1 
driven, skill- 
specific 
tests 



Significant content and 
process gains that in- 
creased with program 
revision and scale-up 



Large effect size for 
content achieve- 
ment and more 
moderate effects 
for process skills. 

(i continued ) 
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Type of 
measure 


Number 

of results Results 


Effect size 


Statewide 


1 Students who completed 


Moderate effect 


tests 


at least one LeTUS unit 


size for all three 




in seventh or eighth 


content areas and 




grade outperformed 


both process 




peers in all content and 


areas. Moderate 




process categories 


effects in increas- 




measured by the test. 


test passing rates. 


Program- 


1 Sixth graders showed 


Large effect size 


designed 


significant overall im- 


for achievement 


measures 


provement on pre- and 


gains pre- and 




post-test measures over 
a four-year period. 


post-test 



The study looking at differences by sex indicated that 
participation in at least one LeTUS unit is associated with 
an apparent reduction in boy-girl achievement differences 
on statewide examinations. 



Constructing Ideas in Physical Science (CIPS) 

Constructing Ideas in Physical Science (CIPS) is an 
inquiry-based, yearlong physical science course that 
attempts to engage seventh or eighth grade middle school 
students in constructing meaningful understanding of 
physical science concepts. The CIPS course is based on the 
themes of interactions and energy transfers between 
objects. CIPS has five units. Each unit consists of two or 
three cycles of activities designed to help students develop 
physics and chemistry concepts. 

Contact 

CIPS Project 

6475 Alvarado Road, Suite 206 

San Diego, CA 92120 

Fax: 619-594-1581 

E-mail: cips@public.sdsu.edu 

Web site: http://cipsproject.sdsu.edu 

Publisher: 

It's About Time, Inc. 

Tel: 888-698-TIME (8463) 

Results 



Type of measure 


Number 
of results 


Results 


Effect size 


Multiple-choice 
content items 


1 


CIPS students scored 
higher than non-CIPS com- 
parisons after controlling 


Small 


Multiple-choice 
process items 




for prior knowledge, stu- 
dent demographics, and 
weeks of instruction in 


Small 


Open-ended 
content items 




Physical Science. 


Moderate 



CIPS participation did not appear to have closed either 
sex (male/ female) or racial (white/ Asian versus non- Asian 
minority) achievement gaps on either multiple-choice con- 
tent or process questions or open-ended content items. 



DESIGNS/DESIGNS II 

Project DESIGNS (Doable Engineering Science Inves- 
tigations Geared for Non-science Students), developed 
by the Harvard-Smithsonian Center for Astrophysics, 
includes design-based activity modules for use in physical 
science and technology education courses in grades 5-9. 
The project's six topics cover chemistry, static forces, elec- 
tricity and magnetism, potential and kinetic energy, energy 
transfer, and force, work, power, and torque. Designs II 
resulted from a follow-up effort to develop a full-year 
middle school (grades 7-9) physical science course based 
on the modules. Project pedagogy was derived from the 
constructivist model of learning and takes into account stu- 
dents' personal theories. The project's goal was to open sci- 
ence concepts to students through activities involving the 
design, construction, and optimization of simple devices. 

Contact 

Program overview: 

Web site: http://cfa- 

www.harvard.edu/sed/resources/designsinfo.html 

Publisher: 

Kendall/Hunt 
Tel: 800-542-6657 

Web site: http://www.kendallhunt. com/index. cfm?PID=219 
&PGI=152 



Results 



Type of measure 


Number 
of results 


Results 


Effect size 


Self-designed 
test to measure 
process skills 


1 


Designs II students gained 
over control students. 


Large 


Self-designed 


1 


Students with initially lower 




tests to measure 

conceptual 

knowledge 


(same 

as 

above) 


scores gain more. 


Large 



Analysis by sex for conceptual knowledge shows no 
significant difference in gains. 



Event-Based Science (EBS) 

The Event-Based Science (EBS) series is a module-based 
program designed for students in grades 6-9, with a focus 
on current events. The series has 18 modules designed to 
last four to six weeks, each focusing on different themes 
and concepts across the domains of earth, life, and physical 
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sciences. The modules may be sequenced over all middle 
school grade levels and combined with other instructional 
materials in order to build a comprehensive middle school 
science program. One or two modules typically are used in 
a year, with teachers selecting particular units based on the 
district's science standards, the local curriculum program, 
the interests of the student population, and their own back- 
ground knowledge in specific topics. EBS is not a "stand 
alone" curriculum. By design, teachers and students sup- 
plement each module with additional data about a specific 
event from various resources included as part of the mate- 
rials, or suggested by the program. 

Contact 

Russ Wright, Project Director 
Montgomery County Public Schools 
850 Hungerford Drive 
Rockville, MD 20850 
Tel: 301-806-7252 
Fax: 301-279-3153 

E-mail: russ@eventbasedscience.com 
Web site: http://www.mcps.k12.md.us/departments/ 
eventscience/ 

Results 

One interim study of the impact of EBS was found. This 
study measures the first three years of the EBS project. The 
report of the final findings from the six-year project is not 
yet available. 

Number 



Type of measure of results Results Effect size 



Curriculum- 1 


In two of the three tested 


Small 


driven, 


years, EBS students out- 




skills-specific, 


performed the control group, 




multiple-choice 


after controlling for prior 




test 


science performance. 




Science attitudes 


EBS students displayed 


Small 


survey 


more positive attitudes 
about science than the 
control group. 




Task-based 


EBS students outperformed 


Moderate 


performance 


the control group, controlling 




assessment rubric 


for prior science performance. 





Expeditionary Learning Outward Bound (ELOB) 

As a whole school reform effort. Expeditionary Learning 
Outward Bound (ELOB), K-12, organizes curriculum, 
instruction, assessment, school culture, and school struc- 
tures around producing high quality student work in 
learning expeditions. These expeditions are long-term, 
in-depth investigations of themes or topics designed 
to engage students both in and beyond the classroom 



through projects, fieldwork, and service. Learning expedi- 
tions are designed with clear learning goals aligned with 
district and state standards. Ongoing assessment is woven 
throughout each learning expedition. 

Contact 

Expeditionary Learning 
100 Mystery Point Road 
Garrison, NY 10524 
Tel: 845-424-4000 
E-mail: info@elob.org 
Web site: http://www.elob.org/ 

Results 

Only one study of many found for ELOB looked at science 
achievement. Two schools are included in this study, but 
only one includes data for science. 





Number 






Type of measure 


of results 


Results 


Effect size 


Standardized 


1 


ELOB students showed 


No effect 


tests 




steady gains in science. 


size reported 



One of the schools in the study experienced an in- 
crease of about 22 percent in immigrant students (who 
were limited English proficient) over the five years 
accounted for in the evaluation. The school showed consis- 
tent gains in all subject areas. This same school also has a 
high number of students qualifying for free and reduced 
price lunch. The evaluation concludes that ELOB, in this 
instance, is "particularly successful with a challenging, 
normally low-achieving population." 



Foundational Approaches 
in Science Teaching (FAST) 

Foundational Approaches in Science Teaching (FAST) is a 
three-volume, comprehensive curriculum program based 
on a constructivist philosophy of learning in which stu- 
dents construct their own knowledge through experiential, 
hands-on learning. Investigations help students build on 
existing knowledge and reinforce conceptual understand- 
ing throughout their work. The continual reinforcement 
and return to concepts allows students to achieve a deep 
understanding of the material and to arrive at that under- 
standing at different points in the curriculum. FAST puts 
the teacher in the role of "director of research." The pro- 
gram emphasizes an instructional strategy that is based on 
the teacher's question development practices and other 
techniques that encourage students to think critically. It is 
heavily laboratory-based, with most concepts taught 
through laboratory experiences where students develop 
skills in measurement and lab procedures. 
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Contact 

Donald Young, CRDG Director 
The University of Hawaii 

The Curriculum Research & Development Group, 
Science Section 

1776 University Avenue, UHS Building 2, Room 202 

Honolulu, HI 96822-2463 

Tel: 808-956-4969 

Fax: 808-956-6730 

E-mail: crdg@hawaii.edu 

Web site: http://www.hawaii.edu/crdg/FAST.pdf 



Results 



Type of measure 


Number 
of results 


Results 


Effect size 


Laboratory skills 


2 


FAST students outper- 


No effect 


Process skills 


1 


formed non-FAST compar- 


size reported 


Knowledge skills 


1 


ison group. 




Statewide test 


1 


In one of two years tested, 


No effect 


scores 




FAST students scored sig- 
nificantly higher than the 
state average, after control- 
ling for student background. 


size reported 


Standardized 


2 


FAST students outper- 


No effect 


test scores 




formed non-FAST students 
in one study. In the second, 
both FAST and non-FAST 
groups scored above na- 
tional norms. 


size reported 



Full Option Science System 
for Middle School (FOSS) 

The Full Option Science System (FOSS) was developed by 
the Lawrence Hall of Science, University of Califomia- 
Berkeley. Funded by the National Science Foundation, FOSS 
combines science content and process with a goal of increas- 
ing scientific literacy for students and instructional effi- 
ciency for teachers. The curriculum is organized into topical 
units, called courses, under each of three strands: Earth and 
Space Science, Life Science, and Physical Science and 
Technology. Each course is an in-depth unit requiring 9 to 12 
weeks of instruction. The units have approximately 
10 investigations, each with three to seven parts. The system 
advocates that students should learn important scientific 
concepts and develop the ability to think well if they are 
engaged in situations where they actively construct ideas 
through their own explorations, applications, and analyses. 

Contact 

FOSS Project 

Lawrence Hall of Science, University of California 
Berkeley, CA 94720 



Phone: 510-642-8941 
Fax: 510 642-7387 
Email: foss@berkeley.edu 
Web sites: http://www.lhsfoss.org/ 

http://www.delta-education.com/science/foss/index. shtml 

Results 

Two studies of FOSS materials met our criteria, though in 
each of these, use of FOSS was included as only one part 
of a larger curriculum or professional development 
intervention. 

A 2003 evaluation of the NSF-sponsored Project In- 
quiry, a broad professional development intervention 
including the adoption and use of FOSS materials, revealed 
that fifth grade students of teachers trained in kit use per- 
formed significantly higher on both multiple-choice and 
constructed /open-ended response assessments than did the 
control group. Self-reported hours of teacher professional 
development were also positively associated with science 
achievement. The authors conclude that professional devel- 
opment is associated with better implementation of inquiry- 
based instruction and greater science topic coverage, which 
is in turn associated with science achievement. 

A 2002 study of the use of multiple kit- and inquiry- 
based science materials (including FOSS) in fourth and 
sixth grade classes for English language learners revealed 
a positive relationship between years in the science pro- 
gram and standardized science test scores. This relation- 
ship remained after controlling for the students' increasing 
English language proficiency. 



Global Lab Curriculum (GLC) 

The Global Lab Curriculum (GLC) was a four-year project 
to create a science course emphasizing student collabora- 
tive inquiry. Organized around six units, GLC culminates 
in the design and conduct of original student investiga- 
tions. The structure purposefully provides substantial 
guidance to students in initial investigations and gradu- 
ally peels away the support to allow students to exercise 
acquired skills in designing and conducting their own 
collaborative experiments. GLC is also designed to cap- 
italize on the Internet as both a communication and moti- 
vational tool that helps establish a cross-cultural science 
community. 

Contact 

Harold McWilliams 

TERC Center for Earth and Space Science Education 
2067 Massachusetts Ave. 

Cambridge, MA 02140 
Tel: 617-547-0430 

E-mail: harold_mcwilliams@terc.edu 
Web sites: http://CESSE.terc.edu 
http://globallab.terc.edu 
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Results 

Number 

Type of measure of results Results Effect size 

Self-designed 1 Students with low levels No effect 
tests of prior knowledge benefit size reported 

more from GLC than those 
with medium or high levels 
of prior knowledge. 



Great Explorations in Mathematics 
and Science (GEMS) 

Great Explorations in Math and Science (GEMS) is a 
supplemental enrichment program for students from 
preschool through eighth grade. GEMS provides teachers 
with more than 70 teacher's guides, support documents, 
and pedagogical handbooks; professional development 
opportunities; an active web site; and a national support net- 
work of GEMS leaders and associates and over 45 regional 
sites. GEMS uses generally accessible materials that inte- 
grate science and mathematics. The program's units, pre- 
sented as flexible enhancements or in curriculum sequence, 
are designed to help all teachers reach all students and fea- 
ture clear, step-by-step teacher instructions. In addition to 
the specific standards-based learning goals and content, the 
program emphasizes cooperative learning and problem- 
solving, literature and language arts connections, and real 
world relevance. GEMS units feature an inquiry-based, 
guided-discovery, student-centered approach to learning. 

An assessment component is in place for the entire series. 

Contact 

Jacqueline Barber 

Lawrence Hall of Science #5200 

University of California, Berkeley 

1 Centennial Drive, Berkeley, CA 94720-5200 

Tel: 510-642-7771 

Fax: 510-643-0309 

Contact e-mail: jbarber@uclink4.berkeley.edu 
Program e-mail: gems@berkeley.edu 
Web site: http://www.lhs.berkeley.edu/GEMS/ 

Results 

Several studies of the GEMS units, which serve grades 
K-8, have been produced. Only one of these met our crite- 
ria and focused on the fourth through eighth grade astron- 
omy unit Earth, Moon, and Stars. 



Number 

Type of measure of results Results Effect size 

Content-specific, 3 Students receiving GEMS No effect 
skills-driven test instruction had signifi- size reported 

cantly higher pre-post 
gains than control groups 
without GEMS instruction. 



High Schools that Work (HSTW) 

High Schools that Work (HSTW) is a whole-school, 
research- and assessment-based reform that offers a frame- 
work of goals and key practices for improving the acade- 
mic, technical, and intellectual achievement of high school 
students. HSTW blends traditional college-preparatory 
content with quality technical and vocational studies. 
HSTW provides technical assistance and staff develop- 
ment focused on techniques and strategies such as team- 
work, applied learning, and project-based instruction. The 
HSTW assessment is based on the National Assessment of 
Educational Progress (NAEP). The developer does not 
offer specific subject area programs, but consultants pro- 
vide workshops customized to fit an individual school's 
needs. 

Contact 

Gene Bottoms, Senior Vice President 
Southern Regional Education Board 
592 10th St., NW 
Atlanta, GA 30318 
Tel: 404-875-9211, ext. 249 
E-mail: gene.bottoms@sreb.org 

Web site: http://www.sreb.org/programs/hstw/hstwindex.asp 

Results 

Two studies were conducted using the same measure. 



Type of measure 


Number 
of results 


Results 


Effect size 


HSTW 

assessment 


1 


Gains in achievement 
increase with the number 
of students in a school 
completing the HSTW 
curriculum. 


No effect 
size reported 


HSTW 

assessment 


1 


Overtime, more students 
met achievement goals 
and completed program. 


No effect 
size reported 



Integrated Mathematics, Science, 
and Technology Curriculum (IMaST) 

Integrated Mathematics, Science, and Technology Cur- 
riculum (IMaST) was developed by the Center for Math- 
ematics, Science, and Technology at Illinois State University. 
IMaST is a standards-based, integrated curriculum for 
grades 6-8. IMaST integrates technology, science, and math- 
ematics and includes connections to the language arts and 
social sciences, as well as readings that profile typical 
careers related to the curriculum content. The curriculum is 
based on the constructivist learning theory that allows con- 
cept development to take place in a structured venue. 
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Contact 

Dr. Karen Lind, Director 

Center for Mathematics, Science, and Technology 

Illinois State University 

Campus Box 5960 

Normal, IL 61790-5960 

Tel: 309-438-3089 

Fax: 309-438-3592 

E-mail: cemast@ilstu.edu 

Web site: http://www.ilstu.edu/depts/cemast/programs/ 
imast.shtml 



Results 



Type of measure 


Number 
of results 


Results 


Effect size 


TIMMS sub-test 


1 


IMaST students outper- 
form traditional peers, 
especially regarding 
science processes. 


No effect 
size reported 



Issues, Evidence, and You (IEY)/SEPUP 

The University of California-Berkeley Lawrence Hall of 
Science developed the SEPUP IEY curriculum for middle 
school and junior high use with support from the 
National Science Foundation. Issues, Evidence, and You 
(IEY) focuses on environmental issues in a social context. 
The program builds upon earlier SEPUP modules and is 
designed to address students' increasing ability to think 
abstractly and builds upon students' need for peer inter- 
action and support. The developer intended the curricu- 
lum to serve as the physical science component of an 
integrated science program (physical, life, and earth sci- 
ence) or as a year-long physical science program. The 
course consists of 65 activities or investigations presented 
in a conceptual sequence. The instructional times of the 
activities vary from one to three class periods. The cur- 
riculum can accompany Science and Life Issues (SALI), 
another SEPUP program now in the piloting phase. The 
program also integrates student assessment into the 
curriculum, providing teachers with a basis for under- 
standing gaps in student knowledge of core scientific 
concepts and a plan for addressing such gaps as they are 
identified. 

Contact 

Web site: http://www.sepup.com/index.htm 
Publisher: 

LAB-AIDS 



17 Colt Court 
Ronkonkoma, NY 11779 
Tel: 800-381-8003 
Fax: 631-737-1286 

E-mail: customerservice@lab-aids.com 

Results 

An assessment of the pilot implementation of IEY was con- 
ducted as an experimental study using pre- and post-tests 
of student knowledge for two groups: one taught using 
IEY and another using the traditional science curriculum. 
The study spanned 15 multi-school sites in 12 states, cov- 
ering 26 classrooms and a total of 830 students. Student 
achievement was measured by the assessments from the 
IEY program, which consisted of multiple choice and short 
answer items. 



Number 

Type of measure of results Results Effect size 

Curriculum-driven, 1 IEY students experienced Moderate 
skill-specific tests an increase in ability to 

present an evidence-based 
line of reasoning, while 
comparison classes did 
not experience significant 
improvement in this skill. 



Learning by Design (LBD) 

Developed by Georgia Tech's EduTech Institute, Learning 
by Design (LBD) is an approach to science learning in 
which middle school students learn as a result of collab- 
oratively engaging in design activities and reflecting 
appropriately on their experiences. Scaffolding is a key 
component of LBD; interventions combine teacher facili- 
tation, paper-and-pencil design diaries and other paper 
prompts, and software tools and prompts. LBD has units 
in both physical and earth science. 

Contact 

The EduTech Institute 
Georgia Institute of Technology 
801 Atlantic Drive 
Atlanta, Georgia, 30332-0280 
Tel: 404-894-3807 

Web site: http://www.cc.gatech.edu/edutech/projects/ 
lbdview.html 
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Results 



Number 

Type of measure of results Results Effect size 



Self-designed 1 


LBD gains in content learn- 


No effect 


test incorporat- 


ing higher than comparison 


size reported 


ing some 


students. Typical LBD stu- 




standardized test 


dents do as well or better 




items (NAEP, 


than honors-level non-LBD 




TIMSS) 


students. 




Performance 2 


Typical LBD students score 


No effect 


assessments 


higher than typical com- 
panion group students in 
applying collaborative 
science skills and prac- 
tices. Typical LBD student's 
performance was on par 
with non-LBD honors stu- 
dents, and LBD honors stu- 
dents outperformed non- 
LBD honors students. 


size reported 



Modeling Instruction in High School Physics 

Modeling Instruction in High School Physics is grounded 
in the thesis that scientific activity centers on modeling: the 
construction, validation, and application of conceptual 
models to understand and organize the physical world. 

The program uses computer models and modeling to 
develop the content and pedagogical knowledge of high 
school physics teachers and train them to be leaders in 
science teaching reform and technology infusion. The pro- 
gram relies heavily on professional development work- 
shops to equip teachers with a teaching methodology. 
Teachers are trained to develop student abilities to make 
sense of physical experience, understand scientific claims, 
articulate coherent opinions of their own, and evaluate evi- 
dence in support of justified belief. For example, students 
analyze systems using graphical models, mathematical 
models, and pictorial diagrams called system schema. 

Contact 

David Hestenes 

Director, Modeling Instruction Program 

Department of Physics and Astronomy 

Arizona State University 

P.O. Box 871504 

Tempe, AZ 85287-1504 

Tel: (480) 965-6277 

E-mail: Hestenes@asu.edu 

Web Site: http://modeling.la.asu.edu/modeling-HS.html 



Results 

Four evaluations of the impact of Modeling Instruction on 
student achievement were found. 



Number 

Type of measure of results Results Effect size 



Tests of alternate 2 
methods of phys- 
ics instruction 

(Comparison of average 
classroom scores be- 
tween teachers using 
Modeling Instruction, 
traditional methods, and 
reform methods) 



Average pre-post test gains No effect 
in Modeling Instruction size 

classrooms were double reported 

those in traditional class- 
rooms, and 10 percentage 
points higher than those in 
reform-method classrooms. 



Tests of alternate 2 



Post-test scores after a 



No effect 



methods of phys- 
ics instruction 

(Comparison of average 
classroom scores 
between teachers pre- 
Modeling Instruction 
training and the same 
teachers post-training) 



teacher is trained in Modeling size 
Instruction were between 6 reported 
and 10 percentage points 
higher than their classroom 
averages before training. 



Data from three studies show that male students con- 
sistently outperform female students. 



National Science Curriculum 
for High Ability Learners 

The National Science Curriculum for High Ability 
Learners Project is a supplemental program that has been 
implemented across grades 2-8 with a broad group of stu- 
dents within the average-to-gifted range of ability. The 
curriculum units employ problem-based learning for 
engaging students in the study of the concept of systems, 
specific science content, and the scientific research process. 
Students engage in a scientific research process that leads 
them to create their own experiments and design their 
own solutions to each unit's central problem. The units 
encourage in-depth study, and content areas cover a 
breadth of scientific subject matter drawn from the physi- 
cal, life, and earth sciences. Each unit constitutes approxi- 
mately 30 hours of instruction, with students typically 
receiving two units within an academic year. Major com- 
ponents of the program include a curriculum framework 
that contains goals and learning outcomes linked to indi- 
vidual lesson plans; embedded and post assessments that 
focus on science content, concept, and process learning; 

25 lesson plans that address these goals and outcomes 
with relatively equal emphasis on each of the goals; and a 
real world problem that serves as the catalyst for subse- 
quent learning in the unit. 
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Contact 

Joyce VanTassel-Baska 
Center for Gifted Education 
College of William and Mary 
427 Scotland St. 
Williamsburg, VA 23185 
Tel: 757-221-2362 
Fax: 757-221-2184 
E-mail: jlvant@wm.edu 
Web Site: http://cfge.wm.edu 



Results 





Number 




Effect 


Type of measure 


of results 


Results 


size 


Open-ended 


1 


Students in National Science 


High 


assessment to 




Curriculum (NSC) classrooms 




test gifted 




scored better on one unit 




science students 




tested when compared to stu- 
dents in non-NSC classrooms. 




Open-ended 
assessment to 
test gifted 
science students 


4 


Same as above 


High 



Physics Resources and Instructional Strategies 
for Motivating Students (PRISMS) 

The goal of Physics Resources and Instructional Strategies 
for Motivating Students (PRISMS) is to provide learning 
activities to promote understanding of physics principles 
in the context of experiences relating to the daily lives of 
secondary school students. PRISMS includes a guide with 
over 130 activities in the form of student instructions and 
teacher notes with background information on the activi- 
ties. The program's resources include several videotapes 
from which students make observations and take data, 
and recommended software and interfacing for schools 
that have access to microcomputers. A complete student 
evaluation and testing program is included in a three- 
to four-diskette set. The instructional strategies blend 
exploratory activities, concept development, and applica- 
tion activities to stimulate problem-solving skills and the 
understanding of major physics concepts. The guide can 
be integrated with the use of any physics textbook and is 
designed to be individualized to meet the needs of each 
teacher. The guide also contains activities dealing with sci- 
entific, technological, and social issues as well as career 
information. The new version is called PRISMS PLUS. 

Contact 

Roy D. Unruh, Director 
Physics Department, Room 303 



University of Northern Iowa 

Cedar Falls, IA 50614-0150 

Tel: 319- 273-2380 

Fax: 319-273-7136 

E-mail: roy.unruh@uni.edu 

Web Site: http://www.prisms.uni.edu/ 



Results 



Type of measure 


Number 
of results 


Results 


Effect size 


State 

assessment 


2 


PRISMS students have 
higher achievement gains 
than comparison students. 


No effect 
size reported 


Program- 

designed 

measures 


1 


PRISMS students' gains in 
reasoning and problem- 
solving skills were greater 
than those of comparison 
students. 


No effect 
size reported 



Science 2000/Science 2000+ 

Science 2000+ (previously known as Science 2000) is a mul- 
timedia, multiyear science curriculum for high schools that 
takes an integrated, thematic approach to the earth, life, 
and physical sciences. At each grade level, the yearlong 
course includes four nine-week units, connected by central 
themes and a storyline — a narrative that sets a real-world 
context for the science content. Each unit poses problems 
related to real-life scientific and social issues. Students 
address these problems by drawing information from CD- 
ROM-based resources (text, images, video, simulations) 
supplemented by laser disc, web references, and manipu- 
lative kits. Multimedia resources (text, images, and video) 
are cross-referenced and linked within the CD. 

Contact 

Ellen M. Nelson 

Decision Development Corp. 

2303 Camino Ramon, Suite 220 

San Ramon, CA 94583-1389 

Tel: 800-835-4332 or 925-830-8896 

Fax: 925-830-0830 

E-mail: info@ddc2000.com 

Web site: http://www.ddc2000.com 



Results 



Type of measure 


Number 
of results 


Results 


Effect size 


Self-designed 
test instruments 


1 


All Science 2000 students 
showed gains in content 
knowledge from pre- to 
post-test. 


No effect 
size reported 
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Study showed the curriculum to be effective in 
increasing the content knowledge of all students, regard- 
less of gender, ethnicity, or language classification. 



Science and Technology Concepts 
for Middle Schools (STC/MS) 

Science and Technology Concepts for Middle Schools 
(STC/MS) is a modular program composed of 24 units. 
There are four units for each grade level, one each in the 
following strands: life science, earth science, physical 
science, and technology. Each STC unit generally has 
16 lessons with hands-on investigations. Teachers can use 
the four modules to comprise the science curriculum for 
the entire school year or use one or two individual units as 
supplements to other curriculum pieces. Eight modules for 
grades 7 and 8 are currently under development. When 
completed, STC/MS will include eight units for science in 
grades 7 and 8. The instructional units will be balanced 
among life, earth, physical sciences, and technological 
design. The components are designed to be offered as two 
one-year courses (one unit from each of the scientific 
strands) or as four single semester courses. 

Contact 

National Science Resources Center 
Arts and Industries Building, Room 1201 
900 Jefferson Drive, SW 
Washington, DC 20560-0403 
Tel: 202-357-3364 
Fax: 202-633-9136 

Web site: http://www.nsrconline.org/curriculum_resources/ 
middle_school.html 

Publisher: 

Carolina Biological 
Tel: 800-334-5551 

Web site: http://www.carolina.com 

Results 

In a 2001 study of four of the eight STC /MS modules, a 
post-test only design was used to compare the perfor- 
mance between groups receiving STC /MS instruction and 
those who received traditional instruction. The impact of 
the curriculum on student achievement in each particular 
content area was measured by multiple-choice and short- 
answer tests developed to measure concepts specific to 
each unit, including TIMMS and NAEP. When possible, 
test items were taken from previously existing assessments 
so that national and international comparisons were possi- 
ble. In all four STC/MS curriculum units, students demon- 



strated significantly higher performance than control 
groups and national / international comparison groups. 

The quasi-experimental evaluation design, however, 
makes it difficult to control for prior knowledge or instruc- 
tion of students in comparison groups. 



The Science Curriculum 
Improvement Study (SCIS) 

The Science Curriculum Improvement Study (SCIS) 
was developed at the Lawrence Hall of Science at the 
University of California between 1962 and 1974 for use in 
grades K-6. The goal of the program is the development of 
scientific literacy, defined as a combination of basic knowl- 
edge concerning the natural environment, investigating abil- 
ity, and curiosity. The program consists of 12 units, one life 
and one physical science unit at each elementary grade level. 
About 10 major concepts are developed each year. The con- 
cepts are interrelated and are intended to provide a concep- 
tual framework for the child's thinking. Opportunities are 
provided for developing science processes as well. The gen- 
eral instructional pattern is free exploration of new materials, 
the introduction of a new concept, and the application of the 
new concept in a range of new situations. 

Contact 

Note: Science Curriculum Improvement Study (SCIS) is now called 
SCIS 3+ 

Publisher: 

Delta Education 
Tel: 800-258-1302 

Web Site: http://www.deltaeducation.com/scisgallery.aspx? 
collection=N&menulD=1 1 

Results 

A 1983 meta-analysis of 57 controlled studies of SCIS and 
two other activity-based science programs (ESS and SAPA) 
draws conclusions about the programs across process, con- 
tent, and affective outcomes. Seventy percent of the stud- 
ies were dissertations, and a conservative estimate of the 
combined students tested is 13,000 in over 900 classrooms. 
Seventy-nine percent of the studies had a quasi-experi- 
mental design. Forty-eight percent of the studies tested 
effects after more than one year of program use. 

The overall effect of these three programs on all outcome 
areas was positive, though not dramatically so; thirty-two 
percent of comparison studies had statistically significant 
results favoring the treatment group, while 
6 percent favored the nontreatment group. These results sup- 
port an overall conclusion of a positive program effect. The 
mean effect size for all studies, with all outcomes weighted 
equally, was .35, or a 14 percentile point improvement over 
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non-activity-based instruction. The effects on measures of 
science process, intelligence, and creativity were positive. 
Small positive effects were observed in attitudes towards 
science. Contrary to a common worry about activity-based 
programs, content achievement was not negatively affected. 

A second meta-analysis conducted in 1986 largely con- 
firmed these results, emphasizing in particular the differ- 
ences in attitude toward science and process skills (17 and 
19 percentile point gains, respectively) among students 
who were taught with activity-based modules. 



World Watcher/Learning about the 
Environment Curriculum (LATE) 

The World Watcher /Learning about the Environment Cur- 
riculum (LATE) is a yearlong, inquiry-based, technology- 
supported environmental science curriculum for high school 
developed at Northwestern University. It is based on the 
Leaming-for-Use model of learning that conceptualizes con- 
tent and process learning as complementary and mutually 
facilitating, rather than at odds. LATE incorporates the use of 
scientific visualizations and is centered on three key issues: 
population growth and resource availability, electricity gen- 
eration and energy demand, and managing water resources 
for agriculture and human consumption. 



Contact 
Daniel C. Edelson 
Northwestern University 
Learning Sciences Program 
Annenberg Hall 
Evanston, III 60208-2610 
E-mail: geode@letus.northwestern.edu 
Web site: http://www.worldwatcher.northwestern.edu/ 
curriculum.htm 



Results 



Type of measure 


Number 
of results 


Results 


Effect size 


Self-designed 


1 


Gains at all grade levels 


Large 


test 




and all populations (urban, 


(urban), 






suburban). Biggest differ- 


moderate 






ences in 10th grade. 


(suburban) 



Urban students showed higher gains (5 points) than 
suburban students (3.5 points). Researchers caution not to 
interpret this result as greater effectiveness of the curricu- 
lum for the urban group. It is possible that a subgroup of a 
sample that scores below average on a test tends to do bet- 
ter on retests. 

The views expressed here are those of the authors and do not necessarily reflect 
those of the Urban Institute, its trustees, or its funders. 
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